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BACKGROUND OF THE INVENTION 

The present invention relates generally to systems for monitoring movement of a 
human eye, and more particularly, to systems for monitoring fatigue of a person based 
upon movement of their eyelid or for controlling electronic equipment using movement 
of the eyelid or by tracking the user's eye. 

Drowsiness and fatigue while driving is a serious problem that leads to 
thousands of automobile crashes each year. Critical aspects of driving impairment 
associated with sleepiness include reaction time, attention, and information processing. 
Sleep related crashes are most common in young people, shift workers whose sleep is 
disrupted by working at night or working long or irregular hours, and people with 
untreated sleep apnea syndrome and narcolepsy. 
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There are some in-vehicle systems that are intended to measure sleepiness or 
some behavior associated with sleepiness in commercial and noncommercial driving. 
Examples include brain wave monitors, eye-closure monitors, devices that detect 
steering variance, and tracking devices that detect lane drift. For example, infrared 
cameras or other devices have been used to monitor a driver's awareness. These 
devices are typically mounted on the dashboard or roof of the driver's vehicle. Such 
devices, however, require that the driver maintain constant eye contact with the camera. 
In addition, they do not monitor eyelid movement if the user looks sideways or turns 
around. These cameras may also have difficulty seeing through eyeglasses or 
sunglasses and may not operate effectively in sunlight. 

In addition to monitoring drowsiness of drivers, such systems are also needed 
for monitoring pilots or other individuals. One system that is independent from the 
vehicle is an eye movement monitoring system which is contained within a frame worn 
on a user's head. An example of this type of system is disclosed in U.S. Patent 
Application Publication No. US2001/0028309 Al, by William C. Torch. The system 
includes an array of emitters for directing light towards a person's eye and an array of 
sensors for detecting light from the array of emitters. The sensors detect light that is 
reflected off of the eye or eyelid, thus producing output signals indicating when the eye 
is covered by the eyelid. The emitters project a reference frame towards the eye, and a 
camera on the frame monitors movement of the eye relative to the reference frame. 
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Such systems, however, are typically large, complex, and costly to manufacture. 
Furthermore, these systems may obstruct at least a portion of the user's view. 

Eye monitoring systems are also used to track a user's eye to control electrical 
devices such as a computer input device (e.g., pointer or mouse) and for assistive 
technology for disability products. There are many different techniques which can be 
used for eye tracking. One conventional system is a remote mounted tracking device. 
A major drawback to this type of system is that it is not very effective when the user is 
actively moving their head or body independent of gazing at a point. Conventional 
head mounted eye tracking devices often require complex optics and processing 
equipment and are typically expensive to manufacture, heavy, making them difficult to 
wear for extended periods of time, and often obstruct at least a portion of a user's view. 
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SUMMARY OF THE INVENTION 



A system and method for detecting eye movement through optical observation 
of the eye are disclosed. The system generally comprises a frame configured to be worn 
on a user, a light source, and at least one eyepiece connected to the frame. The eyepiece 
has a reflective surface configured for reflecting light emitted from the light source onto 
the eye. A sensor is connected to the frame and is operable to detect light reflected by 
the eye in a direction that is substantially parallel to the light incident on a retina of the 
eye to detect that the eye is in an open position. 

In another aspect of the invention, a system for monitoring eye movement 
generally comprises a frame configured to be worn on a user, a light source, and at least 
one eyepiece connected to the frame. The eyepiece has a hot mirror surface on a side 
facing the user's eye when the frame is positioned on the user. The hot mirror surface 
of the eyepiece is configured for reflecting light emitted from the light source onto the 
eye. The system further includes a sensor operable to detect light reflected by the hot 
mirror surface to detect whether the eye is in an open or closed position. 

In yet another aspect of the invention, a system for tracking eye movement 
generally comprises a frame configured to be worn on a user, a light source, and at least 
one eyepiece connected to the frame. The eyepiece has a reflective surface forming a 
hot mirror on a side facing the user's eye when the frame is positioned on the user and 
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the reflective surface of the eyepiece is configured for reflecting light emitted from the 
light source onto the eye. The system further includes a camera connected to the frame 
for monitoring movement of the eye to sense a direction of gaze of the user's eye. 

A method for detecting eye closure through optical observation of the eye 
generally comprises positioning an optical system on a user, the system comprising a 
frame having a reflective surface, a light source, and a sensor. Light is emitted onto the 
reflective surface and reflected towards an eye of the user. The eye operates as a retro- 
reflector and light reflected from the eye is detected at the sensor to determine whether 
the eye is in an open or close position. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a schematic illustrating one embodiment of a system of the present 
invention shown mounted relative to a user's eye. 

Fig. 2 is a perspective of an optical device of the present invention mounted on 
an eyeglass frame. 

Fig. 3 is a perspective of the optical device and eyeglass frame of Fig. 2 
mounted on a user's head. 

Fig. 4 is a perspective of the optical device mounted on a helmet worn on a user. 

Fig. 5 is a block diagram illustrating electronic components of an optical system 
of the present invention. 

Fig. 6 is a flowchart illustrating operation of the optical system. 

Corresponding reference characters indicate corresponding parts throughout the 
several views of the drawings. 
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DETAILED DESCRIPTION OF THE INVENTION 



The following description is presented to enable one of ordinary skill in the art 
to make and use the invention. Descriptions of specific embodiments and applications 
are provided only as examples and various modifications will be readily apparent to 
those skilled in the art. The general principles described herein may be applied to other 
embodiments and applications without departing from the scope of the invention. Thus, 
the present invention is not to be limited to the embodiments shown, but is to be 
accorded the widest scope consistent with the principles and features described herein. 
For purpose of clarity, details relating to technical material that is known in the 
technical fields related to the invention have not been described in detail. 

The present invention is directed to a system for monitoring eye movement, and 
more specifically, the opening and closing of the eye or tracking of the eye to determine 
the direction of a user's gaze. Monitoring of the opening and closing of the eye may be 
used to detect drowsiness or sleepiness based on, for example, the number of blinks, 
time between blinks, or duration of eyelid closure. As described below, the system uses 
light to measure the frequency and speed of a human eye blink. Humans typically blink 
about 5-30 times per minute, with each blink lasting about 200-300 milliseconds. As a 
person gets tired or sleepy, eye blinks get longer and slower until the eyes begin to close 
for short periods. The system disclosed herein may be used, for example, by drivers, 
pilots, operators of heavy machinery, or other individuals to warn them when they are 
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getting tired. The system may also be used in a laboratory environment to study 
individuals with sleep disorders or for medical research. Monitoring of the eye opening 
and closing eye may also be used to control electronic equipment, as described below. 
Tracking the gaze of the user's eye may be used alone or in combination with the 
monitoring of eyelid position to provide additional control of computers or other 
electronically controlled equipment. 

Referring now to the drawings, and first to Fig. 1, a system of the present 
invention is schematically shown with an exemplary eye 20 (including retina, sclera, 
iris, lens, and cornea) of a user positioned relative to the system. The system includes 
an optical device 10 which is attached (either removably or fixedly) to a conventional 
pair of eyeglasses (Figs. 2 and 3). As shown in Figs. 2 and 3, the eyeglasses include a 
frame 14 and a pair of eyepieces (lenses) 16, at least one of which forms part of an 
optical path, as described below. The optical device 10 is preferably attached to a side 
member of the frame 14 and configured to be substantially unobtrusive to the user's 
view. 

As shown in Fig. 4, the frame may comprise other user mounted equipment, 
such as a helmet 15. The helmet 15 includes a visor 17 which forms the eyepiece 
having the reflective surface 18 formed thereon. Only a portion of the visor 17 may be 
formed with a reflective surface, as required to reflect light transmitted from a light 
source onto the eye 20 and transmit light reflected from the eye to a sensor (described 
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below). The remaining portion of the visor 17 may be transparent or tinted to reduce 
light intensity. 

The systems shown in Figs. 2, 3, and 4 include an optical device 10 positioned 
to monitor only one of the user's eyes 20. It is to be understood, however, that the 
system may also be configured with two optical devices 10, one for each eye 20. 

The system is relatively small and lightweight as compared to conventional eye 
movement detection systems. Since the system is preferably contained within the frame 
14 or helmet 15, the data obtained from the optical device 10 is not dependent upon 
movement of the user or the direction the user is looking. 

The system includes optical device 10 and a reflective surface 18, which is 
formed by a surface of the eyepiece 16 positioned opposite the eye 20 (Fig. 1). The 
optical device 10 includes a light source 22, beamsplitter 24, sensor 26, and focusing 
optics (lens) 28. In one embodiment, the light source 22 is a Light Emitting Diode 
(LED) which emits an infrared beam. The emitted light is preferably within the infrared 
range (or near infrared range) to minimize interference with the user's vision since the 
infrared light is not visible to the user. The beamsplitter 24 is positioned to transmit at 
least some of the incident light emitted from the light source 22 to the reflective surface 
1 8. One or more lenses 28 may be positioned to direct light received from the 
beamsplitter 24 onto reflective surface 18, as indicated by light ray 40a. 
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The reflective surface 18 of the eyepiece 16 receives light emitted from the light 
source 22 and reflects the light towards the eye 20 as light ray 40b. The reflective 
surface 18 may be formed by a dielectric or other type of coating. In one embodiment, 
the reflective surface 18 is a hot mirror. The hot mirror selectively separates infrared 
light from visible light. The infrared light is reflected by the hot mirror and light in the 
visible spectrum is transmitted through the surface 1 8. The eyepiece 16 may have 
various curvatures and is preferably shaped to provide optimum performance based on 
the approximate distance between the eyepiece and the eye 20 and optical device 10, as 
is well known by those skilled in the art. 

The light reflected by the eyepiece 16 onto the eye 20 is then reflected by the 
eye which acts as a retro-reflector. When light enters pupil 30 of eye 20, reflection 
takes place at the retina 32 so that the eye acts as a retro-reflector returning a beam 
along the path of the illuminating beam (light rays 40c and 40d). The light rays travel 
after reflection by the retina 32 in a direction that is parallel to that of the source of the 
incident light. The light thus travels the same or substantially the same path, but in the 
opposite direction to, respective incident light rays impinging on the retina. Light exits 
the eye 20 (ray 40e) and is reflected by surface 18 back towards the optical device 10 
(ray 40f). 

Sensor 26 is provided to detect light reflected from the eye 20. The light is 
reflected from the eyepiece 16 back to sensor 26, which is mounted on the frame 14. 
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Due to differences in the reflective characteristics of the surface of the eye and the 
eyelid, the intensity of the light reflected off of the eye depends on whether the eye is 
open or closed. The eyelid does not reflect substantial light back to the sensor 26 since 
it does not operate as a retro-reflector, as does the eye when the eyelid is open. Light 
intensity signals corresponding to eyelid movement received from the sensor 26 are sent 
to a processor, which converts the received data into a suitable signal. The light 
intensity signals may be compared with a predetermined light intensity level to 
determine whether the eyelid is in an open or closed position. The sensor 26 may be, 
for example, a photodiode, photocell, photodetector, camera (described below), or other 
suitable device. One or a plurality of sensors 26 may be used. 

It is to be understood that the system shown in Fig. 1 is only one example and 
that other optical components, such as lenses, filters, reflectors, wave plates, apertures, 
etc., may be used or the arrangement of components may be changed, without departing 
from the scope of the invention. 

Electronic circuitry and components of the system include a processor 30, power 
supply 32, controller 34, transmitter 36, timer 37, alarm 38, and other appropriate 
electronics required to transmit a signal detected based on eyelid position and provide 
proper operation of the optical system (Fig. 5). The controller 34 may be configured to 
control the intensity of light emitted by the light source 22, adjust the threshold 
sensitivity of the sensor 26, turn, the light source on and off, or control the alarm 38. 
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The controller 34 preferably controls the light source 22 to produce a signal pulsed at a 
predetermined frequency (e.g., 4-20 pulses per second). The sensor 26 may be 
synchronized to detect light pulses only at the predetermined frequency. The timer 37 
provides a reference time for comparing the length of time between eyelid movement to 
a predetermined threshold. The timer function may also be provided by a clock of the 
processor 30. The power supply 32 provides power to the light source 22, sensor 26, 
processor 30, alarm 38, transmitter 36, and other components requiring a power source. 
The power supply 32 may be a conventional battery or any other suitable power source. 
Memory (not shown) may also be provided to store the data or signals for further 
analysis. The system may also include one or more digital filters as required to reduce 
noise. 

The processor 30 is coupled to the sensor 26 for converting light intensity 
signals (e.g., binary code) corresponding to eyelid movement received from the sensor 
into data that can be processed and sent to the alarm 38. The processor 30 is configured 
to receive and process an output signal, such as a light intensity signal, from the sensor 
26. The processor 30 may also include circuitry for the transmission of data and 
conversion of the data into various formats. Data may be sent from the transmitter 36 
via a wire attached to the device and leading from the frame 14, 15 to a remote device 
(e.g., unit worn by user or unit mounted adjacent to or remote from the user) or wireless 
transmission may be used to send signals generated by the sensor 26 or processor 30 to 
a remote device using suitable frequencies. The transmitter 36 may be coupled to an 
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amplifier if needed to send the signal to a remote device. The transmitter 36 may not be 
required if the alarm 38 is contained within the optical device 10 and coupled to the 
processor 30 or sensor 26, and if there is no need to send the data to a remote device for 
monitoring or analysis. 

The alarm 38 may be activated when the sensor 26 detects eyelid closure for a 
minimum specified duration, when the frequency of eye blinks increases above a 
specified threshold, or the time between blinks reaches a predetermined limit. The 
alarm 38 may be initiated, for example, when a user begins blinking slower than normal 
for a specified period of time. The alarm 38 may provide an audible alarm, visual 
warning light, vibrating device, or any other suitable device which stimulates one or 
more senses of the user. The alarm 38 may be mounted directly on the optical device 
10 or frame 14, 15 to eliminate the need for remote transmission of data or signals, as 
previously discussed. 

It is to be understood that the block diagram of the circuitry described above and 
shown in Fig. 5 is only one example and that different components and arrangement of 
components may be used, as is well known by those skilled in the art. Also, many of 
the functions shown independently in Fig. 5 may be grouped together as one or more 
components. 

Fig. 6 is a flowchart illustrating an example of operation of the optical system 
described above. The light is first emitted from the light source 22 at step 50. The light 
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is transmitted from the light source to the reflective surface 18 of the eyepiece 16 (step 
52) and reflected from the eyepiece 16 to the eye 20 (step 54). The light is then 
reflected by the eye which acts as a retro-reflector (step 56). The reflective surface 18 
reflects the light towards the sensor 26 at step 58. The sensor 26 transmits a signal 
based on light intensity to the processor 30 (step 60). If the eye movement detected by 
the sensor 26 results in a selected characteristic (e.g., number of blinks, time between 
blinks, duration of eyelid closure) exceeding a predetermined threshold, the alarm 38 is 
activated (steps 62 and 64). The monitoring may then continue after the alarm is 
activated. If the eye movement does not result in an alarm condition, monitoring is 
continued (steps 62 and 66). 

The system may also be configured to track eye movement to determine the 
direction of gaze of the user, in addition to or in place of tracking the opening and 
closing of the eyelid. The direction of gaze of the user's eye 20 or position of the eyelid 
may be used to provide input to a computer (e.g., mouse, pointer, scroll), control on/off 
functions for various electronic devices, or for assistive technology for disability 
products (e.g., wheelchair). For example, a pointer may be placed at a specific location 
on a computer screen by looking at a desired location. A click of a mouse may be 
provided with an eye blink or by staring (dwell) or instructions may be provided by 
monitoring the number of blinks. The system may also be used as a communication 
device. For example, data from the system may be sent to a device which converts the 
data into an audio output. The system may be configured to track the gaze of one or 
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both eyes 20 or may track eyelid movement of one eye and the gaze of one eye, or any 
combination thereof. 

In order to track the gaze of the user's eye 20, a camera is provided as the sensor 
26. The camera may be a charge-coupled device (CCD) that provides an output image 
of the eye 20, which is processed by an image processor to determine the user's 
direction of gaze. The camera is electrically coupled to a computer for digitization and 
processing of the digitized image data. The position of the reflected image of the light 
source 22 is monitored relative to the center of the user's eye, as is well known by those 
skilled in the art. The image processing is used to interpret the position of the eyeball as 
observed by the camera. The position of the eyeball is identified by tracking the 
position of the pupil. One of the eye's visible features may also be used, such as the 
boundary between the sclera and iris or the outline of the pupil against the iris. 

Although the present invention has been described in accordance with the 
embodiments shown, one of ordinary skill in the art will readily recognize that there 
could be variations made to the embodiments without departing from the scope of the 
present invention. Accordingly, it is intended that all matter contained in the above 
description and shown in the accompanying drawings shall be interpreted as illustrative 
and not in a limiting sense. 
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